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CLAIMS 

1. A method for image retrieval using a statistical bigram correlation model, 
the method comprising: 

receivmg a plurality of images responsive to multiple search sessions; 

determining whether the images are semantically relevant images via 
relevance feedback; and 

estimating a respective semantic correlation between each of at least one 
pair of the images with a respective bigram frequency, each respective bigram 
frequency being based on multiple search sessions in which each image of the pair 
is indicated to be a semantically relevant image. 

2. A method as recited in claim 1, further comprising: 

assigning a respective ranking score to each of the images based at least in 
part on the respective semantic correlation correspondmg to the image; and 
displaying only those images with a highest range of ranking scores. 

3. A method as recited in claim 1, furttier comprising, responsive to a search 
session, dynamically updating the respective bigram frequency corresponding to 
two of the images. 

4. A method as recited in claim 1, wherein the respective semantic correlation 
is: (a) a positive correlation between two semantically relevant images; (b) a 
negative correlation between a semantically relevant image and a semantically 
irrelevant image; and (c) no correlation otherwise. 
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5. A method as recited in claim 1 : 

wherein the respective semantic correlation is performed offline or online 
to calculate unigram and bigram frequencies from relevance feedback information, 
the unigram frequency being based on relevance feedback to a session of the 
multiple search sessions, the unigram frequency indicating that each respective 
image of the images is either semantically relevant to the session, semantically 
less relevant to the session, or a non-feedback image with respect to the session; 
and 

wherein each respective bigram frequency is based on a pair of unigram 
frequencies. 

6. A method as recited in claim 1, wherein estimating tiie respective semantic 
correlation fiirther comprises: 

associating a respective unigram frequency with each of the images, the 
unigram frequency indicating that each respective image of the images is either 
semantically relevant, semantically less relevant, or a non-feedback image, the 
unigram frequency being based on relevance feedback to a session of the multiple 
search sessions; and 

wherein each respective bigram frequency is based on a pair of unigram 
frequencies. 
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7. A method as recited in claim 1, wherein estimating the respective semantic 
correlation further comprises: 

associating a respective unigram frequency with each of the images, the 
unigram frequency indicating that each respective image of the images is either 
semantically relevant, semantically less relevant, or a non-feedback image, the 
unigram frequency being based on relevance feedback to a session of the muhiple 
search sessions; 

determining a maximum frequency from a maximum value of the bigram 
and unigram frequencies; and 

wherein the respective semantic correlation is further based on the 
maximum frequency. 

8. A method as recited in claim 1, further comprising identifying, for each 
image obtained responsive to one or more search sessions of the multiple search 
sessions, a respective semantic support based on a similarity measure and/or the 
respective semantic correlation, the similarity measure corresponding to a 
similarity of a respective feature vector of the image and a search query 
corresponding to the session. 
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9. A method as recited in claim 1, further comprising: 
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identifVinff, for each ima&e obtained resDonsive to one or more search 
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sessions of the multiple search sessions, a respective semantic support based on a 
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similarity measure and/or the respective semantic correlation, the similarity 
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measure corresponding to a similarity of a respective feature vector of the image 
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and a search query corresponding to the session; 
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assi^nins a resnective ranking score to each of the ima&es based unon the 
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respective similarity measure, the respective semantic support, and a semantic 
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weight; and 
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scores. 
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10. A method as recited in claim 1 wherein estirnatinpr the rpsneetive spmpntir 
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unigram frequency of image LT is the maximum frequency, R{hJ) is the 
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correlation between image I and J. 
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11. A method as recited in claim 1, wherein each respective bigram frequency 
is based on a pair of unigram frequencies, and wherein the method further 
comprises performing the respective semantic correlation offline by: 

(a) initializing all unigram and bigram frequencies to zero; 

(b) clustering search sessions with a same query into groups; 

(c) calculating unigram counts within a group; 

(d) updating unigram frequencies ; 

(e) updating bigram frequencies; 

(f) repeating operations (c), (d), and (f) for all session groups; 

(g) setting all negative unigram and bigram frequencies to zero; and 

(h) calculating each respective semantic correlation based on results of (a) 
through (f). 
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12. A method as recited in claim 1, wherein each respective bigram frequency 
is based on a pair of unigram frequencies, wherein C(/) is a unigram count of 
image /, and wherein the method further comprises performing the respective 
semantic correlation offline by: 

(a) initializing C(I) to zero (0); 

(b) iteratively updating C(I) for every session in a group such that: 

C(/) = C(/) + 1 , if image / is labeled as relevant in a session; 
C(/) = C(/)-l, if image / is labeled as irrelevant in a session; and 
C(/) is unchanged otherwise. 

(c) repeating (b) for every image of the images; 

(d) updating each respective unigram frequencies as U{I) = U{I) + C(/) ; 

(e) updating each respective bigram frequency of an image pair such that: 

B{I,J) = 5(7, J) + min{C(/),C(J)}, if C(/) > 0,C(J) > 0, 
B{I,J) = B{I,J) - min{C(/) -C(J)} , if C(/) > 0,C(J) < 0, 
3(1, J) = B{I,J) - min{-C(/),C(J)} , if C(/) < 0,C(J) > 0, and 
B{I,J) = B{I,J) , otherwise; and 

wherein /, ^ are two images, B{I,J) is their bigram frequency, and 
U{I) is the unigram frequency of image /. 
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13. A method as recited in claim 1, wherein each respective bigram frequency 
is based on a pair of unigram frequencies, and wherein the method further 
comprises performing the respective semantic correlation online by: 

(a) calculating unigram counts in a particular search session; 

(b) updating unigram frequencies; 

(c) updating bigram frequencies; and 

(d) updating each respective semantic correlation between each of the 
images based on results of (a) through (c). 
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14. A method as recited in claim 1, wherein each respective bigram frequency 
is based on a pair of unigram frequencies, wherein C(/) is a unigram count of 
image /, wherein U{I) is a unigram frequency of image /, wherem 5(7, J) is a 
bigram frequency of image / and J, wherein a session group comprises a single 
search session, and wherein the method fiirther comprises performing the 
respective semantic correlation online by: 

(a) responsive to determining that there is a user log, updating calculating 
each respective unigram and bigram frequency according to data in tiie user log; 

(b) responsive to determining that there is not a user log, initializing each 

C(7;and5f2;to zero(O); 

(c) iteratively updating C(I) for the single search session such that: 

C(/) = 1 , if image / is labeled as relevant; 

C(/) = -1 , if image / is labeled as irrelevant; and 

C(/) = 0 , if C(I) is a non-feedback image; 

(d) updating each respective unigram frequencies as U{I) = U(I) + C(I) ; 

(e) updating each respective bigram frequency of an image pair such that: 

B(I, J) = B(I, J) + 1 , if C(/) > 0, C(J) > 0, 
B(I, J) = 5(7, J) - 1 , if C(7) > 0, C(J) < 0, 
5(7, J) = 5(7,/) -1 , if C(7) < 0,C(/) > 0, or 
B(I,J) = B{I,J), otherwise; and 

wherein 7, J are two images, and B{I, J) is their bigram frequency. 
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15. A computer-readable medium for image retrieval using a statistical bigram 
correlation model, the computer-readable medium comprising computer- 
executable instructions for: 

receiving a plurality of images responsive to multiple search sessions; 

determining whether the images are semantically relevant images via 
relevance feedback; and 

estimating a respective semantic correlation between each of at least one 
pair of the images with a respective bigram frequency, each respective bigram 
frequency representing a probability of whether two of the images are 
semantically related to one-another based on a co-occurrence frequency that each 
image of the two images was relevant in a previous query/feedback session. 

16. A computer-readable medium as recited in claim 15, fiirther comprising 
instructions for: 

assigning a respective ranking score to each of the images based at least in 
part on the respective semantic correlation corresponding to the image; and 
displaying only those images with a highest range of ranking scores. 

17. A computer-readable medium as recited in claim 15, ftirther comprising 
instructions for, responsive to a search session, dynamically updating the 
respective bigram frequency corresponding to two of the images. 
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18. A computer-readable medium as recited in claim 1 5, wherein the respective 
semantic correlation is: (a) a positive correlation between two semantically 
relevant images; (b) a negative correlation between a semantically relevant image 
and a semantically irrelevant image; and (c) no correlation otherwise. 

19. A computer-readable medium as recited in claim 1 5 : 

wherein the respective semantic correlation is performed offline or online 
to calculate unigram and bigram frequencies from relevance feedback information, 
the unigram frequency being based on relevance feedback to a session of the 
multiple search sessions, the unigram frequency indicating that each respective 
image of the images is either semantically relevant to the session, semantically 
less relevant to the session, or a non-feedback image with respect to the session; 
and 

wherein each respective bigram frequency is based on a pair of unigram 
frequencies. 

20. A computer-readable medium as recited in claim 15, wherein estimating 
the respective semantic correlation fiirther comprises instructions for: 

associating a respective unigram frequency with each of the images, the 
unigram frequency indicating that each respective image of the images is either 
semantically relevant, semantically less relevant, or a non-feedback image, the 
unigram frequency being based on relevance feedback to a session of the multiple 
search sessions; and 

wherein each respective bigram frequency is based on a pair of unigram 
frequencies. 
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21. A computer-readable medium as recited in claim 15, wherein estimating 
the respective semantic correlation further comprises instructions for: 

associating a respective unigram frequency with each of the unages, the 
unigram frequency indicating that each respective image of the images is either 
semantically relevant, semantically less relevant, or a non-feedback image, the 
unigram frequency being based on relevance feedback to a session of the multiple 
search sessions; 

determining a maximum frequency from a maximum value of the bigram 
and unigram frequencies; and 

wherein the respective semantic correlation is further based on the 
maximum frequency. 

22. A computer-readable medium as recited in claim 15, further comprising 
instructions for identifying, for each image obtained responsive to one or more 
search sessions of the multiple search sessions, a respective semantic support 
based on a sunilarity measure and/or the respective semantic correlation, the 
similarity measure corresponding to a similarity of a respective feature vector of 
the image and a search query corresponding to the session. 
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23. A computer-readable medium as recited in claim 15, further comprising 
instructions for: 

identifying, for each image obtained responsive to one or more search 
sessions of the multiple search sessions, a respective semantic support based on a 
similarity measure and/or the respective semantic correlation, the similarity 
measure corresponding to a similarity of a respective feature vector of the image 
and a search query corresponding to the session; 

assigning a respective ranking score to each of the images based upon the 
respective similarity measure, the respective semantic support, and a semantic 
weight; and 

displaying only those images with a highest range of respective ranking 

scores. 

24. A computer-readable medium as recited in claim 15, wherein estimating 
the respective semantic correlation is determined as follows: 

• 0<i?(/,J)<l 

• R(I,J) = RiJ,I); 

• if I=JmdUiI)<0; R(I, J) = 0; 

• if I ^ J and Bil, J) <0: R(I, J) = 0; 

• if/=-^and > 0 : r{i, j) = U {I) I T ; or 

• if I * J md B(I,J) > 0 : R{l,j)= B(I)/T ; 

wherein I, J axe two images, B(I,J) is their bigram frequency, U(I) is the 
unigram fi-equency of image I,T is the maximum frequency, RihJ) is the 
correlation between image I and J. 
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25. A computer-readable medium as recited in claim 15, wherein each 
respective bigram frequency is based on a pair of unigram frequencies, and 
wherein the computer-executable instructions further comprise instructions for 
performing the respective semantic correlation offline by: 

(a) initializing all unigram and bigram frequencies to zero; 

(b) clustering search sessions with a same query into groups; 

(c) calculating unigram counts within a group; 

(d) updating unigram frequencies ; 

(e) updating bigram frequencies; 

(f) repeating operations (c), (d), and (f) for all session groups; 

(g) setting all negative unigram and bigram frequencies to zero; and 

(h) calculating each respective semantic correlation based on results of (a) 
through (f). 
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26. A computer-readable medium as recited in claim 15, wherein each 
respective bigram frequency is based on a pair of unigram frequencies, wherein 
C(/) is a unigram count of image /, and wherein the computer-executable 
instructions further comprise mstructions for performing the respective semantic 
correlation offline by: 

(a) initializing C(I) to zero (0); 

(b) iteratively updating C(I) for every session in a group such that: 

C(I) = C(I) + 1 , if image / is labeled as relevant in a session; 

= C(/) - 1 , if image / is labeled as irrelevant in a session; and 
C(/) is unchanged otherwise; 

(c) repeating (b) for every image of the images; 

(d) updating each respective unigram frequencies as U(I) = U(I) + C(I) ; 

(e) updating each respective bigram frequency of an image pair such that: 

B(I,J) = B{I,J) + min{C(/),C(J)} , if C(/) > 0,C(J) > 0, 
J) = B{I,J) - min{C(/),-C(J)} , if C(/) > 0,C(J) < 0, 
B{I,J) = B{I,J) - min{-C(/),C(J)} , if C(/) < 0,C(J) > 0, and 
B{I,J) = B{I,J) , otherwise; and 

wherein I, J are two images, B{I,J) is their bigram frequency, and 
U (/) is the unigram frequency of image /. 
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27. A computer-readable medium as recited in claim 15, wherein each 
respective bigram frequency is based on a pair of unigram frequencies, and 
wherein the computer-executable instructions further comprise instructions for 
performing the respective semantic correlation online by: 

(a) calculating unigram counts in a particular search session; 

(b) updating unigram frequencies; 

(c) updating bigram frequencies; and 

(d) updating each respective semantic correlation between each of the 
images based on results of (a) through (c). 
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28. A computer-readable medium as recited in claim 15, wherein each 
respective bigram frequency is based on' a pair of unigram frequencies, wherein 
C{I) is a unigram count of image /, wherein U{I) is a unigram frequency of 
image /, wherein B{I,J) is a bigram frequency of image / and J, wherein a session 
group comprises a single search session, and wherein the computer-executable 
instructions ftirther comprise instructions for performing the respective semantic 
correlation online by: 

(a) responsive to determining that there is a user log, updating calculating 
each respective unigram and bigram frequency according to data in the user log; 

(b) responsive to determining that there is not a user log, initializing each 
C(7; and 5(7; to zero (0); 

(c) iteratively updating C(I) for the single search session such that: 

C(/) = 1 , if image / is labeled as relevant; 

= -1 , if image / is labeled as irrelevant; and 
C(/) = 0 , if C(I) is a non-feedback image; 

(d) updating each respective unigram frequencies as U{[) = U{I) + C(/) ; 

(e) updatmg each respective bigram frequency of an image pair such that: 

B{I,J) = B{I,J) + \, if C(/) > 0,C(J) > 0, 
B{I,J) = B(I,J) - 1 , if C(/) > 0,C(J) < 0, 
B{I,J) = B{I,J)-\, if C(/) < 0,C(J) > 0, or 
B{I,J) = B(I,J), otherwise; and 

wherein I,J dxt two images, and 5(7, J) is their bigram frequency. 



lee©hayes piic 509.324435$ 



021 102 1144 MS1-880US PA TAPP DOC 



» 

1 

2 
3 
4 
5 
6 
7 
8 

5 9 

^11 

1^12 

ai3 

N=i4 

|15 
16 
17 
18 
19 
20 
21 
22 
23 
24 



29. A computing device for image retrieval using a statistical bigram 
correlation model, the computing device comprising: 

a processor; and 

a memory coupled to the processor, the memory comprising computer- 
executable instructions that are fetched and executed by the processor for: 

receiving a plurality of images responsive to multiple search 

sessions; 

determining whether the images are semantically relevant images via 
relevance feedback; and 

estimating a respective semantic correlation between each of at least 
one pah of the images with a respective bigram frequency, each respective bigram 
frequency being based on multiple search sessions in which each image of the pair 
is mdicated to be a semantically relevant image. 

30. A computing device as recited in claim 29, further comprising instructions 
for: 

assigning a respective ranking score to each of the images based at least in 
part on the respective semantic correlation corresponding to the image; and 
displaying only those images with a highest range of ranking scores. 

31. A computmg device as recited in claim 29, further comprising instructions 
for, responsive to a search session, dynamically updating the respective bigram 
frequency corresponding to two of the images. 
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32. A computing device as recited in claim 29, wherein the respective semantic 
correlation is: (a) a positive correlation between two semantically relevant images; 
(b) a negative correlation between a semantically relevant image and a 
semantically irrelevant image; and (c) no correlation otherwise. 

33. A computing device as recited in claim 29: 

wherein the respective semantic correlation is performed offline or online 
to calculate unigram and bigram frequencies from relevance feedback information, 
the unigram frequency being based on relevance feedback to a session of the 
multiple search sessions, the unigram frequency indicating that each respective 
image of the images is either semantically relevant to the session, semantically 
less relevant to the session, or a non-feedback image with respect to the session; 
and 

wherem each respective bigram frequency is based on a pair of unigram 
frequencies. 

34. A computing device as recited in claim 29, wherein estimating the 
respective semantic correlation further comprises instructions for: 

associating a respective unigram frequency with each of the images, the 
unigram frequency indicating that each respective image of the unages is either 
semantically relevant, semantically less relevant, or a non-feedback image, the 
unigram frequency bemg based on relevance feedback to a session of the multiple 
search sessions; and 

wherein each respective bigram frequency is based on a pair of imigram 
frequencies. 
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35. A computing device as recited in claim 29, wherein estimating the 
respective semantic correlation further comprises instructions for: 

associating a respective unigram frequency with each of the images, the 
unigram frequency indicating that each respective image of the images is either 
semantically relevant, semantically less relevant, or a non-feedback image, the 
unigram frequency being based on relevance feedback to a session of the multiple 
search sessions; 

determining a maximum frequency from a maximum value of the bigram 
and unigram frequencies; and 

wherein the respective semantic correlation is further based on the 
maximum frequency. 

36. A computing device as recited in claim 29, further comprising instructions 
for identifying, for each image obtained responsive to one or more search sessions 
of the multiple search sessions, a respective semantic support based on a similarity 
measure and/or the respective semantic correlation, the similarity measure 
corresponding to a similarity of a respective feature vector of the image and a 
search query corresponding to the session. 
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37. A computing device as recited in claim 29, further comprising instructions 
for: 

identifying, for each image obtained responsive to one or more search 
sessions of the multiple search sessions, a respective semantic support based on a 
similarity measure and/or the respective semantic correlation, the similarity 
measure corresponding to a similarity of a respective feature vector of the image 
and a search query corresponding to the session; 

assigning a respective ranking score to each of the images based upon the 
respective similarity measure, the respective semantic support, and a semantic 
weight; and 

displaying only those images with a highest range of respective ranking 

scores. 

38. A computing device as recited in claim 29, wherein estunating the 
respective semantic correlation is determined as follows: 

• 0<R{I,J)<1 

• R(I,J) = R(J,I); 

• if /=^and < 0 : J) = 0 ; 

• if I ^ J and B(I, J) ^0: R(I,J) = 0; 

• if ^='^and U (I) > 0: R{ij) = u(I)/T; or 

• if / / and B(I,J) > 0 ; R{l,j) = B(I)/T ; 

wherein I, J are two images, B(I,J) is their bigram frequency, U{I) is the 
unigram frequency of image I,T is the maximum frequency, RiI,J) is tiie 
correlation between image I and J. 
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39. A computing device as recited in claim 29, wherein each respective bigram 
frequency is based on a pair of unigram fi-equencies, and wherein the computer- 
executable instructions further comprise instructions for performing the respective 
semantic correlation offline by: 

(a) initializing all unigram and bigram frequencies to zero; 

(b) clustering search sessions with a same query into groups; 

(c) calculating unigram counts within a group; 

(d) updating unigram frequencies ; 

(e) updating bigram frequencies; 

(f) repeating operations (c), (d), and (f) for all session groups; 

(g) setting all negative unigram and bigram firequencies to zero; and 

(h) calculating each respective semantic correlation based on results of (a) 
through (f). 
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40. A computing device as recited in claim 29, wherein each respective bigram 
frequency is based on a pair of unigram frequencies, wherein C(/) is a unigram 
count of image /, and wherein the computer-executable instructions ftirllier 
comprise instructions for performing the respective semantic correlation offline 
by: 

(a) initializing C(I) to zero (0); 

(b) iteratively updating C(I) for every session in a group such that: 

Ci^) = C(I) + l, if image / is labeled as relevant m a session; 
C(/) = C(/) - 1 , if image / is labeled as irrelevant in a session; and 
C(I) is unchanged otherwise; 

(c) repeating (b) for every image of the images; 

(d) updating each respective unigram frequencies as U{I) = U(I) + C(I) ; 

(e) updating each respective bigram frequency of an image pair such that: 

B(I,J) = 8(1, J) + min{C(/),C(J)} , if C(/) > 0,C(J) > 0, 
8(1, J) = B{I,J) - min{C(/),-C(J)} , if C(/) > 0,C(J) < 0, 
5(7, J) = B{I,J) - min{-C(/),C(J)} , if C(/) < Q,C{J) > 0, and 
B{I,J) = B(I,J) , otherwise; and 

wherein I, J are two images, B(I,J) is their bigram frequency, and 
U (I) is the unigram frequency of image 7. 
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41. A computing device as recited in claim 29, wherein each respective bigram 
frequency is based on a pair of unigram frequencies, and wherein the computer- 
executable instructions further comprise instructions for performing the respective 
semantic correlation online by: 

(a) calculating unigram counts in a particular search session; 

(b) updating imigram frequencies; 

(c) updating bigram frequencies; and 

(d) updating each respective semantic correlation between each of the 
images based on results of (a) through (c). 
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42. A computing device as recited in claim 29, wherein each respective bigram 
frequency is based on a pair of unigram frequencies, wherein C(I) is a unigram 
count of image /, wherein U(I) is a unigram frequency of image /, wherein 
3(1, J) is a bigram frequency of image / and J, wherein a session group comprises 
a single search session, and wherein the computer-executable instructions ftirther 
comprise instructions for performing the respective semantic correlation online by: 

(a) responsive to determining that there is a user log, updating calculating 
each respective unigram and bigram frequency according to data in the user log; 

(b) responsive to determining that there is not a user log, initializing each 
C(7;and5(7;tozero(0); 

(c) iteratively updating C(7) for the single search session such that: 

C(/) = 1 , if image / is labeled as relevant; 

C(I) = -1 , if image / is labeled as irrelevant; and 

C(/) = 0 , if C(I) is a non-feedback image; 

(d) updating each respective unigram frequencies as U(I) = U(I) + C(/) ; 

(e) updatmg each respective bigram frequency of an image pair such that: 

5(7, J) = BiU) + 1 , if C(/) > 0,C(/) > 0, 
B(I,J) = B{I,J)-\, if C(/) > 0,C(y) < 0, 
BiI,J) = B{I,J)-\, if C(/) < 0,C(J) > 0, or 
B{I,J) = B(I,J) , otherwise; and 

wherein /, J are two images, and B(I, J) is their bigram frequency. 
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43. A computing device image retrieval using a statistical bigram correlation 
model, the computing device comprising: 

processing means for: 

receiving a plurality of images responsive to multiple search 

sessions; 

determining whetiier the images are semantically relevant images via 
relevance feedback; and 

estimatmg a respective semantic correlation between each of at least 
one pair of the images with a respective bigram frequency, each respective bigram 
frequency being based on multiple search sessions in which each image of the pair 
is indicated to be a semantically relevant image. 

44. A computing device as recited in claim 43, further comprising means for: 
assigning a respective ranking score to each of the images based at least in 

part on the respective semantic correlation corresponding to the image; and 
displaying only those images with a highest range of ranking scores. 

45. A computing device as recited in claim 43, ftirther comprising means for, 
responsive to a search session, dynamically updating the respective bigram 
frequency corresponding to two of the images. 
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46. A computing device as recited in claim 43, wherein the respective semantic 
correlation is: (a) a positive correlation between two semantically relevant images; 
(b) a negative correlation between a semantically relevant image and a 
semantically irrelevant image; and (c) no correlation otherwise. 

47. A computing device as recited in claim 43 : 

wherein the respective semantic correlation is performed offline or online 
to calculate unigram and bigram frequencies from relevance feedback information, 
the unigram frequency being based on relevance feedback to a session of the 
multiple search sessions, the unigram frequency indicatmg that each respective 
image of the images is either semantically relevant to the session, semantically 
less relevant to the session, or a non-feedback image with respect to tiie session; 
and 

wherein each respective bigram frequency is based on a pair of unigram 
frequencies. 

48. A computing device as recited in claim 43, wherein the processing means 
for estimating the respective semantic correlation ftirther comprises means for: 

associating a respective unigram frequency with each of the unages, the 
unigram frequency indicating that each respective image of the images is either 
semantically relevant, semantically less relevant, or a non-feedback image, the 
unigram frequency being based on relevance feedback to a session of the multiple 
search sessions; and 

wherein each respective bigram frequency is based on a pair of unigram 
frequencies. 
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49. A computing device as recited in claim 43, wherein the processing means 
for estimating the respective semantic correlation further comprises means for: 

associating a respective unigram frequency with each of the images, the 
unigram frequency indicating that each respective image of the images is either 
semantically relevant, semantically less relevant, or a non-feedback image, the 
unigram frequency being based on relevance feedback to a session of the multiple 
search sessions; 

determining a maximum frequency from a maximum value of the bigram 
and unigram frequencies; and 

wherein the respective semantic correlation is further based on the 
maximum frequency. 

50. A computing device as recited in claim 43, further comprising processing 
means for identifying, for each image obtained responsive to one or more search 
sessions of the multiple search sessions, a respective semantic support based on a 
similarity measure and/or the respective semantic correlation, the similarity 
measure corresponding to a similarity of a respective feature vector of the image 
and a search query corresponding to the session. 
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51, A computing device as recited in claim 43, further comprising processing 
means for: 

identifying, for each image obtained responsive to one or more search 
sessions of the multiple search sessions, a respective semantic support based on a 
similarity measure and/or the respective semantic correlation, the similarity 
measure corresponding to a similarity of a respective feature vector of the image 
and a search query corresponding to the session; 

assigning a respective ranking score to each of the images based upon the 
respective similarity measure, the respective semantic support, and a semantic 
weight; and 

displaying only those images with a highest range of respective ranking 

scores. 

52. A computing device as recited in claim 43, wherem the processing means 
for estimating the respective semantic correlation is determined as follows: 

• 0<RiI,J)<l: 

• R(I,J) = RiJ,I); 

• 'iiI=JandU(I)<0: R(I,J) = 0; 

• if I ^ J md B(I, J) <0: R(I,J) = 0; 

• if /=/and > 0 : R{i,j) = U(I)/T ; or 

• if / ^ J and 5(7, J) > 0 : r{i,j) = B(I)/T ; 

wherein I, J are two images, B(I,J) is their bigram frequency, U(I) is the 
unigram frequency of image I,T is the maximum frequency, R(I,J) is the 
correlation between image I and J. 
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53. A computing device as recited in claim 43, wherein each respective bigram 
frequency is based on a pair of unigram frequencies, and wherein the processing 
means further comprise means for performing the respective semantic correlation 
offline by: 

(a) initializing all unigram and bigram frequencies to zero; 

(b) clustering search sessions with a same query into groups; 

(c) calculating unigram counts within a group; 

(d) updating unigram frequencies ; 

(e) updating bigram frequencies; 

(f) repeating operations (c), (d), and (f) for all session groups; 

(g) setting all negative unigram and bigram frequencies to zero; and 

(h) calculating each respective semantic correlation based on results of (a) 
through (f). 
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54. A computing device as recited in claim 43, wherein each respective bigram 
frequency is based on a pair of unigram frequencies, wherein C(/) is a unigram 
count of image /, and wherein the processing means fiirther comprise means for 
performing the respective semantic correlation offline by: 

(a) initializing C(I) to zero (0); 

(b) iteratively updating C(I) for every session in a group such that: 

= C(I) + 1 , if image / is labeled as relevant in a session; 
= C(/) - 1 , if image / is labeled as irrelevant in a session; and 
C(/) is unchanged otherwise; 

(c) repeating (b) for every image of the images; 

(d) updating each respective unigram frequencies as U{I) = U(I) + C(/) ; 

(e) updating each respective bigram frequency of an image pair such that: 

5(7, J) = B(I,J) + min{C(/),C(J)} , if C(/) > 0,C(J) > 0, 
B{I,J) = B(I,J) - min{C(/),-C(J)} , if C(/) > 0,C(J) < 0, 
BiI,J) = B(I,J)-mm{-C(I),C(J)} , if C(/) < 0,C(J) > 0, and 
5(7, J) = 5(7, J) , otherwise; and 

wherein 7,7 are two rniages, B(I,J) is their bigram frequency, and 
Uil) is the unigram frequency of image I. 
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55. A computing device as recited in claim 43, wherein each respective bigram 
frequency is based on a pair of unigram frequencies, and wherein the processing 
means further comprise means for performing the respective semantic correlation 
online by: 

(a) calculating unigram counts in a particular search session; 

(b) updating unigram frequencies; 

(c) updating bigram frequencies; and 

(d) updating each respective semantic correlation between each of the 
images based on results of (a) through (c). 
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56. A computing device as recited in claim 43, wherein each respective bigram 
jfirequency is based on a pair of unigram frequencies, wherein C(/) is a unigram 
count of image /, wherein U(I) is a unigram frequency of image /, wherein 
J) is a bigram frequency of image / and J, wherein a session group comprises 
a single search session, and wherein the processing means further comprise means 
for performing the respective semantic correlation online by: 

(a) responsive to determining that there is a user log, updating calculating 
each respective unigram and bigram frequency according to data in the user log; 

(b) responsive to determining that there is not a user log, initializing each 
C(I)mdB(I) to zero (0); 

(c) iteratively updating C(I) for the single search session such that: 

C(/) = 1 , if image / is labeled as relevant; 

C(/) = -1 , if image / is labeled as irrelevant; and 

C(/) = 0 , if C(I) is a non-feedback image; 

(d) updating each respective unigram frequencies as U{I) = U (/) + C{I) ; 

(e) updating each respective bigram frequency of an image pair such that: 

B(I, J) = B(I, J) + 1 , if C(/) > 0, C(J) > 0, 
B{I, J) = 5(7, J) - 1 , if C(I) > 0, C( J) < 0, 
5(7, J) - B(I, J) - 1 , if C(7) < 0, C( J) > 0, or 
5(7, J) = B(I,J), otherwise; and 

wherein 7, J are two images, and B{I, J) is their bigram frequency. 
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